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DETAILED ACTION 

This is a Final Office Action on the merits. This action is responsive to 
Amendments/Remarks, which was filed on 09/26/2007. 

Claims 1-11, and 13-69 are currently pending in this application. Claim 12 
was previously cancelled. Claims 2, 4-6, 9-10, 13-15, 17-18, 22-25, 28-29, 31, 
35-42, 45-49, 52-59, and 62-66 were previously presented. Claims 3, 7-8, 11,16, 
19-20, 26-27, 30, 32-33, 43, 50, 60, and 67 were original presented. Applicant 
has amended independent claims 1, 21, 34, 44, 51, 61, and 68-69. Effective filing 
date is 03/30/2004, (Assignee: Ricoh). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

Claims 1-11, and 13-69 are rejected under 35 U.S.C. 103(a) as being as 
being unpatentable over Chlu et al US007051271B1 filed 10-03-2000 
(hereinafter Chiu), in view of Coar US 200701 06932A1 Continuation of 
09/533,152 filed 03/23/2000 (hereinafter Coar). 
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Regarding independent claims 1, Chui teaches: (as amended) 

a method in a computer system for creating a composite 
electronic representation including presentation material 
information, 

(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations made on the paper documents during the meeting can be extracted 
and used as indexes to the video. This interpretation is supported by Applicant's 
Specification, which states "creating a composite electronic representation 
comprising the determined information" at Pages 3 Para 15.) 

the method comprising: scanning a paper document to generate an 
electronic presentation of the document with presentation material. 
(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations made on the paper documents during the meeting can be extracted 
and used as indexes to the video. 

Also see Chui at Col. 2, Lines 15-25, discloses a method for linking a 
scanned document to a segment of a video is provided. Whereby the scanned 
document identifier is then compared to the plurality of video frame identifiers; 
the scanned document is linked to a first video frame in the plurality of video 
frames responsive to the comparison step. This interpretation is supported by 
Applicant's Specification, which states "key frame images obtained from video 
information captured during the presentation" at Pages 21 Para 57. ) 
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extracting a visual feature from the electronic presentation of the 
document, the visual feature corresponding to a portion of the 
presentation material; 

(See Chui at the Abstract, teaches annotations made on the paper documents 
during the meeting can be extracted and used as indexes to the video. An 
orthonormal transform, such as a Digital Cosine Transform (DCT) is used to 
compare scanned documents to video frames. 

comparing the feature to the recorded information to determine 
information in the recorded information corresponding to the feature, 
the recorded information including information recorded during a 
presentation of the presentation material, 
(See Chui at Col. 2, Lines 15-25, discloses a method for linking a scanned 
document to a segment of a video is provided. Whereby the scanned document 
identifier is then compared to the plurality of video frame identifiers; the scanned 
document is linked to a first video frame in the plurality of video frames 
responsive to the comparison step. This interpretation is supported by Applicant's 
Specification, which states "key frame images obtained from video information 
captured during the presentation" at Pages 21 Para 57. ) 

accessing recorded information including at least one of audio and 
visual information recorded during a presentation of the presentation 
material, and comparing the visual feature to the recorded 
information to determine a portion of the recorded information 
corresponding to the visual feature, , whereby at least a portion of 
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the recorded information corresponds to a feature portion of the 
presentation material; 

(See Chui at Col. 2, Lines 15-25, discloses a method for linking a scanned 
document to a segment of a video is provided. Whereby the scanned document 
identifier is then compared to the plurality of video frame identifiers; the scanned 
document is linked to a first video frame in the plurality of video frames 
responsive to the comparison step. 

Also see Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 
300 for accessing a segment of video file 200 which references scanned 
document 101a.) 

the computer system thus creating a composite electronic 
representation of the document including the user selectable object, 
the user selectable object being placed in a position associated with 
the extracted feature and allowing the user to access the portion of 
the recorded information in an application displaying the composite 
electronic representation or a separate application by selecting the 
user selectable object; and storing the composite electronic 
representation for access by the user or anther user accessing the 
composite electronic document. 
(See Chui at Col. 2, Lines 15-25, discloses a method for linking a scanned 
document to a segment of a video is provided. Whereby the scanned document 
identifier is then compared to the plurality of video frame identifiers; the scanned 
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document is linked to a first video frame in the plurality of video frames 
responsive to the comparison step. 

Also see Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 
300 for accessing a segment of video file 200 which references scanned 
document 101a. 
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Also see Chui at Fig. 5 and Col. 8 Line 5, discloses memory item 806 and 
persistent storage 808 stores a scanned digital document 101a and video file 
200.) 
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In addition Chui does not teach, but Coar teaches: 

generating a user selectable object providing a user with 
access to the portion of the recorded information corresponding to 
the visual feature, and inserting the user selectable object into the 
electronic representation of the document when the computer 
system locates a portion of the recorded information corresponding 
to the visual feature, 
(See Coar at Fig. 1 and at Para 86, discloses generating a user selectable object 
providing a user with access to the portion of the recorded information 
corresponding to the visual feature (i.e. XWPL (extensible Workflow Package 
Language is an extensible Markup Language (XML) based language. One 
portion of the language defines a standardized method to place information into a 
machine-readable symbol such as a high-density barcode) symbols (one symbol 
for each document and row in the data source) for each document to be 
scanned. The user would then prepare the documents with the XWPL symbol in 
the proper sequence to permit scanning, the user could then scan the 
documents, and the application would then, using the necessary symbol 
recognition tools, extract the data from the symbol, and automatically place the 
image of the document within the correct VirPack in the correct placemen.) 
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FIGURE 1 
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It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify Coar's a user selectable object providing a user with 
access to the portion of the recorded information corresponding to the visual 
feature (i.e. machine readable barcode), so that, when selected by a user the 
barcode would accessing a segment of video file which references scanned 
document as taught by Chui. One of ordinary skill in the art would have been 
motivated to modify this combination because Coar and Chui are from the same 
field of endeavor of electronic presentation includes annotation to presentation 
material, and provides a predictable result of generating a composite electronic 
presentation with a browser interface for accessing a segment of video file which 
references scanned document- see Chui at Fig. 3 and Col. 4, Line 1-15.) 
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Regarding Independent claim 21, (as amended) 

the rejection of claim 1 is fully incorporated. 

In addition, Chui teaches: 

generating composite information based on the portion of in 

the recorded information that corresponding to the visual feature 

and the electronic representation of the document. 
(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations made on the paper documents during the meeting can be extracted 
and used as indexes to the video. Chui further discloses a method for linking a 
scanned document to a segment of a video is provided. Whereby the scanned 
document identifier is then compared to the plurality of video frame identifiers; 
the scanned document is linked to a first video frame in the plurality of video 
frames responsive to the comparison step-See Chui at Col. 2, Lines 15-25, 
discloses.) 

Regarding Independent claim 34, (as amended) 

the rejection of claim 1 is fully incorporated. 

In addition, claim 34 is directed to a computer product to perform the 
method of claim 1, which cites above, and is similarly rejected under the same 
rationale (see claim 1 rejection cites above, and also see Chui at Fig. 1). 
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Regarding Independent claim 44, (as amended) 

the rejection of claim 21 is fully incorporated. 

In addition, claim 44 is directed to a computer product to perform the 
method of claim 21 , which cites above, and is similarly rejected under the same 
rationale (see claim 21 rejection cites above, and also see Chui at Fig. 1) 

Regarding Independent claim 51, (as amended) 
the rejection of claim 1 is fully incorporated. 

In addition, claim 51 is directed to a data processing system to perform the 
method of claim 1 , which cites above, and is similarly rejected under the same 
rationale (see claim 1 rejection cites above, and also Chui at Fig. 1). 

Regarding Independent claim 61, (as amended) 
the rejection of claim 21 is fully incorporated. 

In addition, claim 61 is directed to a data processing system to perform the 
method of claim 21, which cites above, and is similarly rejected under the same 
rationale (see claim 21 rejection cites above, and also see Chui at Fig. 1). 

Regarding Independent claim 68, (as amended) 
the rejection of claim 1 is fully incorporated. 

In addition, claim 68 is directed to a system to perform the method of claim 
1 , which cites above, and is similarly rejected under the same rationale (see 
claim 1 rejection cites above, and also see Chui at Fig. 1). 
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Regarding Independent claim 69, (as amended) 
the rejection of claim 21 is fully incorporated. 

In addition, claim 69 is directed to a system to perform the method of claim 
1, which cites above, and is similarly rejected under the same rationale (see 
claim 21 rejection cites above, and also see Chui at Fig. 1). 

Regarding claims 2, 22, 35, 45, 52, and 62, Chui teaches: 

determining associating information for the recorded 
information that corresponds to the extracted feature. 

(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations made on the paper documents during the meeting can be extracted 
and used as indexes to the video. Chui further discloses a method for linking a 
scanned document to a segment of a video is provided. Whereby the scanned 
document identifier is then compared to the plurality of video frame identifiers; 
the scanned document is linked to a first video frame in the plurality of video 
frames responsive to the comparison step-See Chui at Col. 2, Lines 15-25, 
discloses.) 

Regarding claims 3, 23, 46 and 63, Chui teaches: 

wherein the association information comprises time 
information and source information for recorded information. 
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(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations made on the paper documents during the meeting can be extracted 
and used as indexes to the video. Chui further discloses a method for linking a 
scanned document to a segment of a video is provided. Whereby the scanned 
document identifier is then compared to the plurality of video frame identifiers; 
the scanned document is linked to a first video frame in the plurality of video 
frames responsive to the comparison step-See Chui at Col. 2, Lines 15-25, 
discloses. 

Also see Chui at Fig. 2, illustrates bar 104a represents the time and 
duration in which document 101 is presented as projection 101b in window 203.) 

Regarding claims 4, 36, and 53, Chui teaches: 

associating the association information with the determined 
additional information in the composite electronic representation. 

(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations (i.e. additional information) made on the paper documents during the 
meeting can be extracted and used as indexes to the video.) 

Regarding claims 5, 24, 37, 47, 54 and 64, Chui teaches: 

receiving a selection to the determined additional information 
in the composite electronic representation and using the association 
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information for the additional information to access the recorded 
information. 

(See Chui at Col. 2, Lines 15-25, discloses a method for linking a scanned 
document to a segment of a video is provided. Whereby the scanned document 
identifier is then compared to the plurality of video frame identifiers; the scanned 
document is linked to a first video frame in the plurality of video frames 
responsive to the comparison step. 

Also see Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 
300 for accessing a segment of video file 200 which references scanned 
document 101a.) 

Regarding claims 6, 25, 38 and 55, Chui teaches: 

accessing the recorded information using the determined 
additional information. 

(See Chui at Col. 2, Lines 15-25, discloses a method for linking a scanned 
document to a segment of a video is provided. Whereby the scanned document 
identifier is then compared to the plurality of video frame identifiers; the scanned 
document is linked to a first video frame in the plurality of video frames 
responsive to the comparison step. 

Also see Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 
300 for accessing a segment of video file 200 which references scanned 
document 101a.) 
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Regarding claims 7, and 26, Chui teaches: 

displaying the accessed recorded information. 

(See Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 
101a.) 

Regarding claims 8, and 27, Chui teaches: 
playing the accessed information. 

(See Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 
101a.) 

Regarding claims 9, 29, 39, 49, 56 and 66, Chui teaches: 

performing at least one of emailing, printing, storing, and 
copying the created composite electronic representation. 

(See Chui at Fig. 5 and Col. 8 Line 5, discloses memory item 806 and persistent 
storage 808 stores a scanned digital document 101a and video file 200.) 

Regarding claims 10, 28, 40, 48, 57 and 65, 

Chui does not expressly teach, but Coar teaches: 

determining metadata using the recorded information that 
corresponds to the feature, wherein the composite electronic 
representation includes the metadata. 
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(See Coar at Fig. 1 and at Para 86, discloses generating a user selectable object 
providing a user with access to the portion of the recorded information 
corresponding to the visual feature (i.e. XWPL (extensible Workflow Package 
Language is an extensible Markup Language (XML) based language. One 
portion of the language defines a standardized method to place information into a 
machine-readable symbol such as a high-density barcode) symbols (one symbol 
for each document and row in the data source) for each document to be 
scanned. The user would then prepare the documents with the XWPL symbol in 
the proper sequence to permit scanning, the user could then scan the 
documents, and the application would then, using the necessary symbol 
recognition tools, extract the data from the symbol, and automatically place the 
image of the document within the correct VirPack in the correct placemen. Also 
see Coar at Para 6, describes the metadata.) 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify Coar's a user selectable object providing a user with 
access to the portion of the recorded information corresponding to the visual 
feature (i.e. machine readable barcode; metadata), so that, when selected by a 
user the barcode would accessing a segment of video file which references 
scanned document as taught by Chui. One of ordinary skill in the art would have 
been motivated to modify this combination because Coar and Chui are from the 
same field of endeavor of electronic presentation includes annotation to 
presentation material, and provides a predictable result of generating a 
composite electronic presentation with a browser interface for accessing a 
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segment of video file which references scanned document- see Chui at Fig. 3 
and Col. 4, Line 1-15.) 

Regarding claims 11, 41 and 58, Chui teaches: 

wherein the received electronic representation of the paper 
document includes notes taken by a user, wherein the created 
composite electronic representation includes the notes taken by the 
user. 

(See Chui at Col. 7, Lines 1-10, discloses a method of extract the ink 
annotations, a simple comparison between the original and the annotated paper 
handout may be performed. When more than one set of handouts have been 
annotated, these ink strokes may be extracted and selectively layered over the 
common background of the scanned document. Another way to display the ink 
annotations and notes is simply to show them without a background. In any case, 
the ink strokes may be hyperlinked to play their corresponding segment in the 
video recording.) 

Regarding claim 13, Chui teaches: 

determining a document that includes the recorded 
information using the extracted feature. 

(See Chui at Col. 7, Lines 1-10, discloses a method of extract the ink 
annotations, a simple comparison between the original and the annotated paper 
handout may be performed. When more than one set of handouts have been 
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annotated, these ink strokes may be extracted and selectively layered over the 
common background of the scanned document. Another way to display the ink 
annotations and notes is simply to show them without a background. In any case, 
the ink strokes may be hyperlinked to play their corresponding segment in the 
video recording.) 

Regarding claim 14, 

Chui does not expressly teach, but Coar teaches: 

determining a portion of the document that includes the 
information corresponding to the feature. 

(See Coar at Fig. 1 and at Para 86, discloses generating a user selectable object 
providing a user with access to the portion of the recorded information 
corresponding to the visual feature.) 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify Coar's a user selectable object providing a user with 
access to the portion of the recorded information corresponding to the visual 
feature (i.e. machine readable barcode), so that, when selected by a user the 
barcode would accessing a segment of video file which references scanned 
document as taught by Chui. One of ordinary skill in the art would have been 
motivated to modify this combination because Coar and Chui are from the same 
field of endeavor of electronic presentation includes annotation to presentation 
material, and provides a predictable result of generating a composite electronic 
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presentation with a browser interface for accessing a segment of video file which 
references scanned document- see Chui at Fig. 3 and Col. 4, Line 1-15.) 

Regarding claims 15, 31, 42, 50 and 59, 

Chui does not expressly teach, but Coar teaches: 

an identifier to a location in the recorded information, wherein 

the information in the recorded information corresponding to the 

feature is determined using the identifier. 
(See Coar at Fig. 1 and at Para 86, discloses generating a user selectable object 
providing a user with access to the portion of the recorded information 
corresponding to the visual feature (i.e. XWPL (extensible Workflow Package 
Language is an extensible Markup Language (XML) based language. One 
portion of the language defines a standardized method to place information into a 
machine-readable symbol such as a high-density barcode) symbols (one symbol 
for each document and row in the data source) for each document to be 
scanned. The user would then prepare the documents with the XWPL symbol in 
the proper sequence to permit scanning, the user could then scan the 
documents, and the application would then, using the necessary symbol 
recognition tools, extract the data from the symbol, and automatically place the 
image of the document within the correct VirPack in the correct placemen. Also 
see Coar at Para 6, describes the metadata.) 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify Coar's a user selectable object providing a user with 
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access to the portion of the recorded information corresponding to the visual 
feature (i.e. machine readable barcode; metadata), so that, when selected by a 
user the barcode would accessing a segment of video file which references 
scanned document as taught by Chui. One of ordinary skill in the art would have 
been motivated to modify this combination because Coar and Chui are from the 
same field of endeavor of electronic presentation includes annotation to 
presentation material, and provides a predictable result of generating a 
composite electronic presentation with a browser interface for accessing a 
segment of video file which references scanned document- see Chui at Fig. 3 
and Col. 4, Line 1-15.) 

Regarding claims 16 and 32, 

Chui does not expressly teach, but Coar teaches: 

the identifier comprises at least one of a barcode and 
signature information. 

(See Coar at Fig. 1 and at Para 86, discloses a machine-readable symbol such 
as a high-density barcode.) 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify Coar's a user selectable object providing a user with 
access to the portion of the recorded information corresponding to the visual 
feature (i.e. machine readable barcode; metadata), so that, when selected by a 
user the barcode would accessing a segment of video file which references 
scanned document as taught by Chui. One of ordinary skill in the art would have 
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been motivated to modify this combination because Coar and Chui are from the 
same field of endeavor of electronic presentation includes annotation to 
presentation material, and provides a predictable result of generating a 
composite electronic presentation with a browser interface for accessing a 
segment of video file which references scanned document- see Chui at Fig. 3 
and Col. 4, Line 1-15.) 

Regarding claims 17-18, Chui teaches: 

receiving the electronic representation comprises receiving a 
scan of the document, the document being a paper document, 
wherein receiving the electronic representation comprises 
determining an electronic image of the document, the document 
being a paper document. 

(See Chui at Col. 7, Lines 1-10, discloses a method of extract the ink 

annotations, a simple comparison between the original and the annotated paper 

handout may be performed.) 

Regarding claims 19, and 33, Chui teaches: 

wherein receiving the electronic representation comprises 
receiving the electronic representation in response to an input from 
a user indicating that the composite electronic representation should 
be created. 
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(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. 
annotations made on the paper documents during the meeting can be extracted 
and used as indexes to the video. This interpretation is supported by Applicant's 
Specification, which states "creating a composite electronic representation 
comprising the determined information" at Pages 3 Para 15.) 

Regarding claims 20, 30, 43, 60 and 67, Chui teaches: 

the document comprises a paper document. 

(See Chui at Col. 7, Lines 1-10, discloses a method of extract the ink 
annotations, a simple comparison between the original and the annotated paper 
handout may be performed.) 

It is noted that any citations to specific, pages, columns, lines, or figures in 
the prior art references and any interpretation of the references should not be 
considered to be limiting in any way. A reference is relevant for all it contains 
and may be relied upon for all that it would have reasonably suggested to one 
having ordinary skill in the art. See, MPEP 2123. 
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Response to Arguments 

The Arguments filed on 09/27/2007 has been fully considered but they are 
not persuasive. Applicant's amendment necessitated the new ground(s) of 
rejection presented in this Office action . 

Conclusion 

Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Quoc A. Tran whose telephone number is 
571-272-8664. The examiner can normally be reached on Monday through 
Friday from 9 AM to 5 PM EST. 
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If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Doug Hutton can be reached on 571-272-4137. The fax 
phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). 



/Quoc A, Tran/ 
Patent Examiner 
Art Unit 2176 
12/07/2007 

/(Doug Muttonl 
Doug Hutton 
Supervisory Primary Examiner 
Technology Center 2100 



